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Background / Context: 

Effective communication in writing is one of the most important of all objectives in the 
elementary grades. Among most writing researchers and reformers, there has been a consensus 
over the past 20 years regarding the nature of effective writing instruction. The recommended 
approach is a writing process model that emphasizes teaching children to work in peer response 
groups to help one another plan, draft, revise, edit, and “publish” compositions in various genres 
(Calkins, 1983; Graves, 1983; Hillocks, 1984; Harris & Graham, 1996). The idea is to give 
students insights into the writer’s craft and writing strategies by involving them in a process of 
planning, drafting, revising, and editing and by engaging them with peers, giving them an 
opportunity to learn from each other and learn from the activity of critiquing others’ work 
(Bereiter & Scardamalia, 1987; Case et al., 1994). 

Research on the use of writing process models has generally supported their effectiveness 
in comparison to traditional approaches (Hillocks, 1984; Harris & Graham, 1996), especially 
when it includes explicit strategy instruction (De La Paz & Graham, 2002; Glaser & Brunstein, 
2007; Graham, 2006; Harris, Graham, & Mason, 2006). In particular, research has supported the 
use of a form of writing process called Cognitive Strategy Instruction in Writing (CSIW; Englert 
et al., 1991). In CSIW teachers teach explicit strategies for specific writing genres (such as 
comparison/contrast), “think-alouds” to verbally model thinking about composition, and model 
construction and revision of text. A program called Self-Regulated Strategy Development 
(SRSD), which helps students with learning disabilities think through the writing process and use 
self-regulation and self-reinforcement strategies, has also been found to improve writing 
outcomes (Graham & Harris, 2006). 

The problem with writing process programs is that they are difficult to implement for 
many teachers (Pressley & El-Dinary, 1997). Successful studies of writing process methods have 
generally involved highly motivated teachers in middle-class schools, and most have been very 
small scale, matched comparisons, leaving open the possibility that the teachers who succeeded 
with writing process methods were ones who were already exceptionally interested in writing, 
were already capable teachers, and devoted more time to writing than did control teachers 
(Pressley & El-Dinary, 1997). 

In order to rule out self-selection of teachers as an explanation for positive effects of an 
educational program, it is essential to carry out an experiment in which teachers willing to 
implement the program are assigned at random to experimental or control conditions. Puma 
(2006) carried out such an evaluation of a version of writing process called Writing Wings in 
high-poverty Title I schools. Writing Wings places an emphasis on the cooperative learning 
aspects of writing process, but otherwise trains teachers to teach students a sequence of planning, 
drafting, revising, editing, and “publishing” compositions. Teachers were assigned at random to 
use Writing Wings or to serve in a waiting list control group (i.e., control teachers received 
training after the experiment was over). Third and fourth graders were given writing prompts, as 
pre- and posttests, and these were scored using a holistic scale by scorers unaware of students’ 
treatment assignments. Analyses of covariance, using pretests as covariates, found no differences 
between experimental and control students (Puma, 2006). Observations revealed that teachers 
had a great deal of difficulty in implementing writing process, and some did little teaching of 
writing at all. Given that teachers received considerably more professional development and 



2011 SREE Conference Abstract Template 



1 




coaching than teachers usually do in writing methods, consisting of a day-long workshop and six 
on-site coaching visits, the low levels of implementation were surprising. 

Focus of the Study: 

The present study represented an effort to improve on the outcomes of the Puma (2006) 
study by creating a writing process program that provided students with compelling video 
models of effective writing practices in small writing teams. In this method, called Writing 
Wings with Media (WWM), students worked in 4-member, heterogeneous writing groups to help 
one another plan, draft, revise, edit, and publish compositions, as in the earlier Writing Wings 
program. However, in WWM, students were shown a series of humorous, professionally 
designed puppet skits in which a four-member writing team learns to use writing process 
elements in a variety of genres. The idea was to communicate directly to the students themselves 
(as well as to teachers) a vision of how to work in writing teams, in hopes that this would help 
teachers implement the program with greater fidelity and build enthusiasm and strategic insights 
among students. 

The theory of action for the embedded multimedia aspect of Writing Wings with Media 
focused on the problem of transfer from workshop to classroom (see Joyce, Calhoun, & Hopkins, 
1999; Joyce & Showers, 2002). The idea was that instead of teaching teachers to use writing 
process methods and then hope that they could communicate them to children, the videos would 
go directly to teachers and students at the same time, demonstrating key behaviors and ideas for 
effective writing. 

Setting: 

The study took place in 22 high-poverty schools located in 1 1 states (Florida, Hawaii, 
Texas, Louisiana, Illinois, Mississippi, New Mexico, Washington, Wisconsin, Ohio, and 
Oregon). 

Population / Participants / Subjects: 

There were a total of 46 teachers randomly assigned to WWM and 44 teachers assigned 
to control. Overall, approximately 30% of students were African American, 27% White, 26% 
Hispanic, and 17% Other. 

Intervention / Program / Practice: 

Program elements . The main elements of Writing Wings with Media were as follows. 

Teams. Students were assigned to 4-member learning teams, including high, average, and 
low achievers, boys and girls, and students from any ethnic groups represented in the class. If the 
class did not divide evenly by four, some teams had 5 members. 

Writing process elements. Team members were taught to write compositions using a 
series of steps: 

Plan: Students worked with teammates to plan what they were going to write. 

Draft: Students wrote a draft, or “sloppy copy,” of their composition. 
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Revise: After a partner critiqued the draft (based solely on content, organization, and 
style, not mechanics), the writer wrote a revision. 

Edit: A partner read the revised draft and suggested edits based on grammar, punctuation, 
usage, and spelling. Initially, partners focused on a small set of issues (e.g., capital letters at the 
beginning of each sentence), but as lessons on mechanics skills were presented, these skills were 
added to an editing checklist. 

Publish: After a final review by the teacher and final revisions, students had opportunities 
to present their final compositions to the class, to create a team book or newspaper, or otherwise 
celebrate their writing products in a public forum. 

Multimedia. Students viewed a series of video vignettes illustrating the elements of 
writing process in various genres. A team of puppets, the Write-On Dudes, modeled the process. 
The video team included a student who tended to think she had little to say, one who had trouble 
with organization, one who tended to overwrite, and one who tended to lack detail. In the 
puppets’ interactions, effective cooperative behaviors as well as writing behaviors were modeled, 
and metacognitive strategies adapted from CSIW and SRSD were demonstrated. In addition, 
students viewed a series of live-action skits and animations that presented humorous 
demonstrations of key elements of grammar, punctuation, and usage. 

An example of how media and cooperative learning are used in Writing Wings is 
provided below in a series of lessons on describing events in a sequence. 

Day 1 : A lesson about sequences of events helps students understand how organizing a 
description of an event using a sequence helps the readers’ understanding. In addition, students 
leam to use a graphic organizer for a sequence. 

Day 2: Students work in their teams to brainstonn about events that would be interesting 
to write about and to read about. In an earlier lesson, a video featuring a team of four puppets, 
Ricardo, Mona, Flash, and Tasha, demonstrated peer support for brainstonning, so students are 
practicing modeled skills. Teams share ideas with the class after team discussion. 

Day 3: Students view a video of the puppet team, the Write-On Dudes, using the 
organizer presented on Day 1 to write a plan for their description. Then students write an 
individual plan, review it with their teammates, and revise it based on teammate comments. 

Day 4: Students draft their descriptions. 

Day 5: Students view a video of the Write-On Dudes sharing their drafts, and using a 
“Revision Guide” to evaluate it and make suggestions for revisions. Several students share with 
the class, and the class makes suggestions for revision guided by the teacher. Students then share 
their drafts with their teams, and receive suggestions for revision. 

Day 6: The teacher presents a lesson on using vivid verbs with support from a video 
featuring the Language Mechanics, who repair sentences that are not quite right. Teams practice 
the strategies presented in the video with sample sentences. 

Day 7: Students revise their drafts using feedback from their teammates and strategies 
modeled by the Language Mechanics to use vivid verbs. 

Day 8: The teacher presents a lesson on adding adverbs to enrich description with support 
from a video featuring the Language Mechanics. Students once again revise with an eye toward 
enriching their descriptions with adverbs. 

Day 9: Students work with teammates to edit their writing using an editing checklist 
presented in an earlier lesson featuring the Language Mechanics. Final copies are published and 
shared. 
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Professional development Teachers in Writing Wings with Media received a one-day 
training at the beginning of the program and were then visited by coaches four times over the 
course of the year. Coaches provided feedback and suggestions to help teachers use the program 
elements. 

Research Design: 

This study used an experimental design, with random assignment of volunteer teachers 
within schools. Researchers randomly assigned teachers from each participating grade at each 
school to the treatment (Writing Wings with Media) or control (pre-existing writing program) 
group. After baseline data were collected, schools were informed of which teachers were 
randomly assigned to the treatment and control groups. Random assignments were divulged after 
pretesting to help reduce the likelihood of schools organizing classrooms in a way that might 
affect the equivalence across the groups. 

Data Collection and Analysis: 

Measures. Students were given a pretest in October, 2007 and a posttest in May, 2008. 
They were given one of two writing prompts during both administrations. Prompts were either a 
narrative assignment involving a personal experience, or an infonnative assignment requiring 
students to share knowledge or information about a favorite sport, movie, or book. Prompts were 
randomized within each class of students, during both the pre- and posttests. 

Each essay was “blind coded” so that only the student’s school, incorporated in the code, 
could be identified. Independent raters scored each essay on three different dimensions: Style, 
Ideas and Organization, and Mechanics. Mechanics consisted five subscales, each rated from 0 
(weak) to 2 (strong): Capitalization, punctuation, spelling, complete sentences, and verb usage. 
Ideas and Organization focused on whether or not the student answered the prompt provided, 
organized the response in a logical fashion, used transitions when necessary, and provided 
supporting details to larger ideas. Style focused on the student’s use of evocative words, sentence 
variety, sensory details, and modifiers. Interrater reliabilities were computed as agreements 
divided by agreements + disagreements, averaging 80% agreement for the instrument as a whole. 

Analyses. Ratings were transfonned to z-scores to facilitate combining across grades. A 
multivariate analysis of covariance (MANCOVA) was conducted to determine the effects of the 
treatment on all three dependent variables taken together, controlling for all three pretests. 
Univariate analyses were then carried out for each posttest, controlling for all three covariates. 



Table 1 Here 



Findings / Results: 

MANCOVA results revealed significant differences between the treatment groups on the 
three dependent variables: Wilks’ 7= .990, F(4, 9 1 2)=2.36, p=X)5. Outcomes of the univariate 
analyses are summarized in Table 1. At pretest, the experimental and control groups were very 
well matched on all three measures of writing quality. At posttest, there were small but 
statistically significant differences favoring Writing Wings with Media on ratings of Style 
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(ES=+0.18, p<.01), but there were no significant differences on Ideas and Organization (ES= 
+0.07, n.s.) and only marginally significant differences on Mechanics (ES=+0.12, p<.10). 

Conclusions: 

The findings of this randomized evaluation of Writing Wings with Media indicate small 
positive effects on ratings of students’ compositions at posttest, controlling for pretest measures. 
The magnitude of the gains in effect sizes are modest, ranging from +0.07 to +0. 18, but it is 
interesting to note that the mean gain from third to fourth grade in the control group was only 
+0.13 for Style, +0.22 for Ideas and Orgaization, and +0.29 for Mechanics. That is, the 
significant gains for WWM (vs. control) students on Style were 138% of the gains ordinarily 
made over a whole year of school. The (non-significant) gains for Ideas and Organization were 
equal to 32% of a year’s gain, and the (marginally significant) gains for Mechanics were about 
41% of a year’s gain. These gains were in addition to expected growth during the school year, 
which was represented by control students’ growth. What these comparisons suggest is that 
although the additional gains in writing skills shown by WWM students were small compared to 
effect sizes typical in studies of reading and math interventions, they were substantial in 
comparison to ordinary year-to-year growth in writing skills. The likely reason for this is that 
measurement of gains in writing perfonnance is difficult, and comparing effect sizes from 
studies of reading and math, which are easier to measure, to those in studies of writing skills, 
may be inappropriate. 

It is also important to note that the design of the experiment may have reduced the quality 
of implementation. Because WWM and control classes were in the same schools, opportunities 
for collegial interaction were restricted. Writing Wings teachers were asked not to discuss 
writing instruction with control teachers. The WWM teachers may have implemented less of the 
program as a result, although implementation ratings were consistently higher for WWM than 
they were in the earlier Writing Wings evaluation (Puma, 2006). A cluster randomized 
evaluation of WWM, in which entire schools either implement or do not implement the program, 
might have allowed for both stronger implementations and a sharper contrast between 
experimental and control conditions. 

From a practical perspective, the findings of the study of Writing Wings with Media 
suggest that schools can improve writing outcomes for children in the upper-elementary grades 
using a writing process approach that emphasizes cooperative learning and adds regular video 
demonstrations of the writing process as played out in various genres. Further development and 
evaluation may find more powerful means of advancing students’ writing skills, but the present 
study provides evidence that building on this base is likely to be fruitful. Additional research is 
also needed to determine the unique contributions to outcomes made by the writing process 
approach, cooperative learning, and the use of embedded multimedia. 
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Appendix B. Tables and Figures 



Table 1 

Pre- and Posttest Scores on Writing Measures, 
Grades 3-4 (in z-scores) 





N 


Pre 






Post 








M 


ES 


M 


Adjusted 


ES 






(sd) 




(sd) 


Mean 




Style 














WWM 


467 


0.00 


0.00 


+0.08 


+0.089 


+0.18** 






(1.03) 




(1.02) 






Control 


455 


0.00 




-0.09 


-0.091 








(0.97) 




(0.98) 






Ideas and 














Organization 














WWM 


467 


-0.01 


-0.02 


+0.04 


+0.036 


+0.07 






(0.98) 




(0.98) 






Control 


455 


+0.01 




-0.04 


-0.037 








(1.02) 




(1.02) 






Mechanics 














WWM 


467 


+0.04 


+0.08 


+0.08 


+0.056 


+0. 12 a 






(1.00) 




(1.02) 






Control 


455 


-0.04 




-0.09 


-0.057 








(1.00) 




(0.98) 







a p <.10 

* p<.05 
** pc.Ol 
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